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ABSTRACT 

This  paper  considers  a class  of  variable  metric  methods  for  unconstrained 
minimization  problems.  It  is  shown  that  with  a step  size  of  one  each  member  of 
this  class  converges  locally  and  superl inear ly. 
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SIGNIFICANCE  AND  EXPLANATION 


Many  practical  problems  in  operations  research  may  be  reduced  to  minimizing  a 
function  without  constraints.  Variable  metric  methods  are  successfully  used  in  com- 
putinq  a sequence  which  converges  to  the  minimum  of  a function.  During  each  iteration 
a search  direction  and  a step  size  are  computed.  In  order  to  obtain  fast  convergence 
it  is  necessary  that  the  chosen  step  size  approximates  the  optimal  step  size,  i.e., 
the  step  size  which  minimizes  the  function  along  the  qiven  search  direction.  This 
may  require  considerable  computat iona 1 effort.  If  an  approximation  to  the  minimum 
is  known,  however,  it  is  often  possible  to  increase  the  efficiency  of  an  algorithm 
by  showinq  that  a step  size  of  one  is  a sufficiently  qood  approximation  to  the  opti- 
mal step  size.  Such  a situation  arises,  for  instance,  if  as  in  the  method  of  penalty 
functions  a sequence  of  unconstrained  problems  is  solved  in  order  to  obtain  a solution 
to  a more  complicated  optimization  problem.  In  general  the  minimum  of  one  penalty 
function  is  a good  approximation  to  the  minimum  of  the  penalty  function  used  next. 

In  this  paper  it  is  shown  that  variable  metric  methods  converge  rapidly  with 
a step  size  of  one  if  a good  approximation  of  the  solution  is  used  as  starting  point . 


Accession i c r 


; NTU  C\a.v  1 
DDC  TAS 

1 Ife)  B 

Ju.  ' 

-TT- 

1 

v _ 

□ 

1 By 

Pi  Ctrl  ‘ * • 

A VO  i 1 -;M  Hty  Codes 

i Dist 

A 

/iV.i  I 1 f 

spec 

uiU/oi*  1 

lal  j 

The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive  summary 
lies  with  MKC,  and  not  with  the  author  of  this  report. 


LOCAL  AND  SUPERLINEAR  CONVERGENCE 
OF  A CLASS  OF  VARIABLE  METRIC  METHODS 


Klaus  Ritter 


Introduction 


If  a variable  metric  method  is  used  to  compute  a minimizer  z of  a function  F(x)  it 
simultaneously  generates  a sequence  of  points  {x^}  and  a sequence  of  matrices  (H J.  During 
each  iteration  a correction  is  added  to  lb  with  the  intent  to  construct  an  approximation  to 
the  inverse  Hessian  matrix  of  F(x) . 

A large  class  of  such  methods  has  been  introduced  by  Huang  (8) . A restriction  of  the  Huang 
class  to  update  formulas  which  are  of  rank  two,  satisfy  the  quasi-Newton  equation  and  maintain 
the  symmetry  of  H ^ leads  to  a class  of  methods  proposed  by  Broyden  II]  and  Fletcher  16).  In 
[9]  global  and  superlinear  convergence  has  been  established  for  each  member  of  this  subclass 
without  the  requirement  of  an  optimal  step  size.  However,  in  the  context  of  a qlobal  conver- 
gence theory  the  Broyden-Fletcher-Goldfarb-Shanno-method  {2],  [6],  (7],  110]  appears  to  be  the 
only  one  for  which  it  can  be  shown  that  a step  size  of  one  is  always  acceptable  after  suffi- 
ciently many  iterations. 

Using  a step  size  of  one  Broyden,  Dennis  and  More  [3]  have  shown  that  the  Broyden-Fletcher- 
Goldfarb-Shanno-method  and  the  Davidon-Fletcher-Powell-method  [4] , [5]  converge  superl inearly  to 
a minimizer  z of  F(x)  provided  that  the  initial  point  xQ  and  the  initial  matrix  H^  are 
sufficiently  close  to  z and  the  inverse  Hessian  matrix  of  F(x)  at  z,  respectively.  It  is 
the  purpose  of  this  paper  to  extend  this  result  to  all  members  of  the  Broyden  class. 
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2. 


Preliminary  result*. 


let  x < e"  and  let  Fix)  be  a real-valued  fum t ion.  If  Fix)  is  twice  differentiable 
at  a point  x.  we  denote  the  gradient  and  the  Hessian  matrix  of  Fix)  at  x.  by  g^  * VF(x _.) 
and  G..  * Glx.) , respectively.  A prime  is  used  for  the  transpose  of  a vector  or  a matrix.  For 
any  x t En,  j|x||  denotes  the  Euclidean  norm  of  x. 

Throughout  this  paper  we  require  the  following  assumption  to  be  satisfied. 

Assumption  1 

There  is  a vector  z such  that  Fix)  is  twice  continuously  differentiable  in  some  convex 
neighborhood  of  z,  VF(z)  « 0,  G • G(z)  is  positive  definite  and  the  l.ipschitz  condition 

(2.1)  ||G|x)-G(z)  ||  < 1. 1|  x— z 1 1 , 

where  L is  a positive  constant,  is  satisfied  for  all  x in  some  convex  neighborhood  of  z. 

Clearly  the  above  assumption  implies  that  there  are  constants  0 < u < n and  a convex 
neighborhood  U(z)  such  that,  for  every  x t U(z) , the  inequality  (2.1)  and  the  relation 

VI  ||  y II  2 < y'G(x)y  < T1 II  y II  “ for  all  y , En 

hold. 

We  consider  the  problem  of  determining  a sequence 


(2.2) 


>1 


sj<  i - 0-1.2,... 


which  converges  to  z. 

If  a variable  metric  method  is  used  to  compute  the  sequence  (2.2),  then  an  (n,  n) -matrix 
is  associated  with  each  x.  and  the  search  direction  s^  is  determined  by  the  relation 


3j  ' Vi 


The  matrix  associated  with  x.^,  obtained  by  adding  a rank  two  matrix  to  IP  in 

such  a way  that  ll1  + i satisfies  the  quasi-Newton  equation 

Vidi  ■ pj  • 


where 
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"WIO  1111  ii  i (jpii  i ‘ mu  i 


.\ 


<1*0,.  s 

d - -J-Jil , 
j II* ill  ‘l  fr.J 


Th*  V**iOU*  var‘*bl,!  <*itfe,  the  update  procedure  which  la  used  to  c.»- 

M1«<  f‘"«i  H(.  A Uivje  class  of  such  methods  has  been  studied  by  tUOyden  til,  Kietche. 

lh),  and  Huanu  1*1.  In  the  follow, no  we  will  consider  a subclass  of  these  update  procedure, 
whuh  has  the  property  that  if  the  initial  matrix  H(v  is  sy^etric  all  subsequent  matrices  « 

W‘U  l’e  sv,™”"t,u--  I*  >'«a  shown  ...  |9)  that  the  u, slate  formulas  that  cor.e.pond  to  th.s 

subclass  can  be  written  m the  fo.m 


‘S1  > »l*  ,d*H  d 

II  , n . 1 -i-i-  J-JLl  — ■ L.i  i j „ 

”l  ’ 'VYYS'V^'.V,  JJ 


1’  d‘ll  *11  d p* 

- a -J.J  j l.i  l . 


It  .d  d 1 11  . 


' 1 vvvvsv,  * 2 vsvwvi  ■ 

where  and  tt^  ate  arbitrary  parameters  with  tf2  * it2  > 0. 

T*'  well-known  members  of  th.s  class,  namely  the  Hl\»  - Method  (Btoyden  |J),  rietche.  In), 
ooldfarb  HI,  Shanno  (U'D  and  the  MT  - Method  (llavidon  (41,  Fletcher,  Powell  is,,  can  be  ob- 
ta.ned  by  choos.no  H,  - l.  d,  - 0 and  ^ - 0,  «,  - 1.  resect  ively.  The  choice  tt,  - 
results  ill  the  rank  one  instate  formula 


U'  -»ol  , (pt-d-tl.) 

H , * It  * - 4.  J J 

1*1  ) (p’.-d'H  Id. 

lilt 

lloweve,  th.s  method  >.  known  to  be  numer ically  unst able . U will  be  excluded  the  follow.no. 
It  tti  ,s  po.it.»e  definite  ,t  has  been  shown  in  (4)  that  can  be  written  ,n  the  form 

P ,p!  4 ‘1*  I.  p.  p* 

(*’••»>  it  - — Lx.  * jl'i  t y 'jiiii 

i nift'.r,  w\| . ••  d'  p 

111  l l 1*1  li  H 


II  o 

it  P - .~1J  . c - -L_ 

1 HVill  ' iiViii  1 


- »- 


1 


it)  « »)v»niq  ^ ,q  ^ ^ ) much  that  • 0,  w'q^  * l)  and  q ^ • H^w.  *kdS  n°,n' 

111)  the  vtK'tor*  d,.M..,d  j are  orthoqonal  to  p ami  q ami  are  tuii'h  that 

' ) nl  j ) 


ami 


j;,hAi  • °*  i'k  * * " k • 


p . ■ N.d,.,  i « ) has  norm  one. 

>1  1 »i 


Then  every  H ^ determined  by  (2.11  has  the  form  (see  l ■-> ) > 


p.p*  n u'  II  p i*’ 

„ . , . VM  ♦ « . -M  ♦ v -iXJi  . 
tM  Vi  1 Vi  tit  Vii 


i*i 

n p i»* 

- joij 

i*  i' 
il‘  il 

where  the  vector  is  uniquely  determined  by  the  conditions 

(2.f>)  u.  < spanlq^.p^),  ||u^||  - 1,  d^u^  - 0,  w'u^  > 0 . 

The  parameter  ui^  depends  on  the  particular  numbers  and  h , used  in  (.’.l).  Hole  pie- 

c i ae l y , 


(2.7) 


with 


(2.R1 


1 ‘ Y1llql  ’ d'p.  rj 


<Vir 

H.dU'.rtt, 

Ill  l P.d’d 


1 ViVWVi 


This  shows  that  if  H is  positive  definite  and 


Vrqmpi 

1 * P • ■*  ■ * J-  > 0 . i . •»  . a * 

ri  ||M!  * • W 


Vi 


then  i*  positive  definite  if  and  only  if  » 0. 
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3.  Siypei  l Inear  conve  i 

Throughout  this  Met  ion  we  will  assume  that  Assumption  1 is  satisfied  and  that  the  sequence 
(x^)  u generated  by  the  following  algorithm. 


Algol  1 1 tun 

Step  Oi  v'hoose  numhot  s»  with  rt  ♦ 4 0,  a vector  x , and  a symmetric  i*oiitive 

* — 12  12  0 

definite  matiix  Compu t e g^  - VF(x^) . It  g^  - 0,  stopi  otherwise  set  i * 0 and  go  to 

Step  l. 

Step  l:  Set 


“ MJ9J 


and 


vi 


s 

1 


Compute  9 - VF(*  . 

if  gJH  - o. 

st  v>p  1 

otherwise  go  to  Step  2. 

St  ep  2 1 Comput  e 11  ^ , j by 

(2.1).  Replace 

j 

with 

VI  and  go  to  Step  1 

We  will  show  that  for 

every  choice  of 

*1 

and 

P with  i<|  t d,  f 0 

« - MP,  -P  .) 

and 

yi 

- VISj.S,) 

there  are  minthct  s 


such  that 


« ||x0-*||  « v 


imply  that  every  H ^ is  a well-defined  i>ositive  definite  matiix  and  that  the  algoi  ittim  eithei 
terminates  after  a finite  number  of  iterations  or  generates  a sequence  (x.)  which  converges 
supe (linearly  to  z . 

The  convergence  proof  is  based  on  an  estimate  for  the  trace  of  the  matiix 


l t.  1> 


l "2 

where  G is  the  square  root  of  the  positive  definite  matrix  G,  G 
* li^'.  It  follows  immediately  from  (2.41  and  (2.M  that 


and 


w 


n d d1 


Observe  that  by  (2.6) 


d ,d'.  , w w'.  n d . ,d!  . 

, - -LJ.  + J_  J.J.  + y M.\i 

j+l  d(p . u . w(u . . L,  d ! .p. . 

))  3 3 3 1=3  13*13 


q.+a.p.  -d'.q. 

u » -3.-J  3 . a . .11 

3 |hj+  jPjir  3 d'p. 


Therefore,  we  have  II  q ,+a  .p . w’.u . = w(q.  and  by  (2.7) 
3 fJ"  3 3 3 j 


u.u'.  (q  .ta  .p  . ) (q.+a.p.)  ' 

u -J-J.  = y .„i3,..3.  JL_ 

3 w'.u.  j w'.q. 

3 3 3S3 


, w . w ' , w . w 

1 3 3 _ 1 3 3 


u . w.u.  y.  w'q. 

333  3 33 

Let  denote  the  trace  of  the  matrix  (3.1).  Since  the  trace  of  a matrix  is  equal  to  the 

sum  of  its  diagonal  elements  it  follows  from  (2.4),  (2.5)  and  (3.2)  throuqh  (3.5)  that  we  have 
the  following  relation  between  + ^ and 

p'Gp  +p^g\G  Xg  p(Gp  +d(G  *d 


P3g3P3 


q(Gq.+w'.G  1w  . , y 2 (q  ,+a  .p  . )'G(q  ,+a  .p  . ) +w  (G  ^w. 

. 3^3  3 3 + J_  '3  ^3  3*3  43  3*3 1 1 

w ; q . y . v’.q . 

343  3 33 

/p'.Gp  .+d  '.G  ^d  . \ 

h ‘ • ’) 

. „ . (i . A . fex&S 

'j  w.q.  W , / w.q.  \ p.g.p. 

3 3 '3  / 3 3 \ 3 3 3 


(^i+a.p.)  *G(q  +a  p ) q(Gq 


w’.q.  / 
3 3 / 


In  the  following  five  lemmas  we  will  establish  some  properties  of  i|i  and  the  terms  on  the 
right  hand  side  of  (3.6)  which  will  enable  us  to  prove  the  key  result  that  the  sequence  {iji  t 
is  bounded. 


. ,»•  i*iA 


Umu  1 


1 rt  be  jh'suivo  definite.  Then 


1)  » ^ 2.  -n 


iv)  !Hjlt  \ ♦jllc  1 II H <_  } 


n 

,y  • K be  such  that  y’x  e 0.  Set  v » y-(lx . Then  (see  l'»l) 


M.m  x'GxayV;*1^  _ 2 + v'G~K 

y'x  y’x 

The  first  statement  of  the  lemma  follows  immediately  from  this  equality.  By  definition  y is 

1/2  1/2  -i / > - 1 _ i ,>2 

equal  to  the  sum  of  the  eigenvalues  of  the  twvi  matrices  tl  H.G  * and  G *11  G *.  Sine. 

1 1 

With  mati  ices  are  positive  definite  we  have 

II  Gl  *11. G1  *||  V , and  ||g~'  *H.'g  1 2||  s . 


Hi**  Aimi  It's  t ho  pi  oof  of  t ho  l omm,i . 


lot  M ^ bo  pos  i t i vo  itof  ini  to . For  ovory  0 < X < l there  Ait*  constants  N 2n  And 

* 0 such  that,  for  every  1, 

y . < i t and  |)x  -*  ||  v i* 


i-'Givn-^'.G*1., 


pjvi 


2 i *1  - in 


it  follow*;  fiiMn  (l.?>  anti  l.omnw*  l that 


|iV,-GPjlr  - 0(*j-2til  and  ||s.||  - 0(||q^||) 


Therefore,  and  are  *n  (*(z3  for  t^  and  tJ  sufficiently  small  and  we  obta 

from  Taylor's  theorem  the  relation 


0.10) 


where 


g.  . * g.  - Cs.  - E.s. 
3 + 1 3 3 33 


/ G (x ,-ts . ) dt  - 
n 1 3 


and 

(3.11) 


E.  11  £ max  ))G(x  -ts  ) - G 1 1 
J 0<t<l  1 3 


Therefore , 


< max  ||  L(x  - t (x.  - x ) - z) 
0<t<l  3 3 3 + 1 


< L maxf  ||x.-z||  , ||  x j + 1~z  ||  ) 


(3.12) 


19-i 


i±L_  < 

- 


qj 

N7 


- G 


+ EJ 


3 I 


I S II  !|s.  I 

KlfH0:9]  ' Gp3!l  + IkJ 


Since  Taylor's  theorem  and  Assumption  1 imply 
0.13)  lln  I 


I ' 9 j f I * 0('x.-z||)  and  ||x.-z||  = 0(||g..||)  , 

it  follows  from  (3.9),  (3.11)  and  (3.12)  that  ||x.+1-z||  < X||x.-z||  for  ^ and  xj  suffi- 
ciently small. 


Lemma  3 


Let  H.  be  positive  definite.  Then  there  are  constants  2n  < t < x and  0 < x*  < x* 


2-1 


2 - 1 


such  that,  for  every  j. 


D-j  < t2  and  ||x.-z||  < x* 


imply 


-8- 


l)  H . is  well-defined  and  positive  definite. 

li)  1 - y.  - -2 — * — * 

’ l 


2|  ♦ 0(  Xj-z  ) 


Proof : 

Hy  i2.“>)  ar.  i (2.7),  H is  wel  1-def  ined  and  iositivc  definite  if  d't  J and  i ■ . . 

3*1  Vi  3 

Us  i nq  (3.1 o)  wo  obtain 

d p » p Gp  . ♦ p *.  F . p . 

33  *3  3 3 33 


Since  by  (J.rt)  and  (3.11) 
M.  14) 


* 0(||x.-z||) 


this  shows  that  d’p^  ' 0 for  t,  and  t * sufficiently  small. 
By  (2.4) 


(d'p . ) ( d '. q . ) ' 

d'U.d  ■ ■■  i £■  t‘ i-i 

J 1 3 • rV . wiq3 


Theioforo  it  follows  from  (.'.(3)  that. 


( t.lSI 


dip 


(d’.q.)' 


2 r 

(d;q.)  ! ...  . ... 

l-v  - -H  j. U,  * 6 --  - t & i-i 

1 Vjdjpj  I ejvj  2 wi°idipj 


-1 


Py  do f i n l t ion , 


(3.1c) 


.!•  i 

j 1 


o '.o  . -q  ' , q . , \q‘  , q . 1 II  q . 'I  N q I 

J j 3*1  3 I , 3*1  )‘  " 3 jtl 


*3 


j" 


1^11  ||9}| 


(3.17) 


d;i'  . ,i  j < ,_x  -i-M. 


. ‘i  r 

3 v ) 


Vi 


1 - Vj  !lq j II 


It  follows  from  part  11)  of  lemma  1 that  the  sequences  {l/w'q  1,  (l/d'.p  ( q ' '|s  1 and 

3 3 3 3 3 3 

i q^ll/q’p^  are  bounded.  Because  8j  ♦ 8 , ^ 0 wo  deduce,  therefore,  from  (3.q),  (3.12), 
and  (3.  41  throuqh  (1.17)  that 


( 3. IB) 


11  - - °Ii3f 


2\ 


< 1 


for  and  T*  sufficiently  small.  Furthermore,  usinq  (3.18)  and  (3.12)  we  obtain 

|l  - y^l  ” 0(  II  p j'GPj  II  ~ + I|e.||2> 

which  by  (3.7)  and  (3.14)  implies  the  second  part  of  the  lemma. 


Lemma  4 

Let  H j be  positive  definite  and  0 < t < 1.  There  are  constants  2n  < t^, 

0 < T*  < T * and  > 0 such  that,  for  every  j, 


*j  i t3  a,ld  llxj-zll  i T* 


imply 


p’.Gp.+d'G  *d  . 

i>  J d'-p 1 - 2 < SjXj-zl 

3 3 


(q.+o.pj  'Gtq/a.pJ-q^Gq, 

J j^j 

qtcq,  ! wiG"lwj 
-i— 1 + (—  - 1)  -2-; 2 

u’n  v w ' n 


iii)  (y^  - 1) 


w'q . y . 

j 3 3 


w ,q  . 
3H3 


'p^GPa+p2q^G  1g^ 


. '1  - 2 < 4 II- _ 

V pjgjpj  / “ 


Proof : 


It  follows  from  (3.10)  that 


d . - Cp,  « E.P. 

3 j y 1 


which  by  (3.7)  and  (3.14)  implies  the  first  part  of  the  lemma.  In  order  to  prove  the  second 


part  we  observe  that 


lVaipj>’G(VVV  ' ‘ 2“ipjCi‘,i  * y'i1^ 

■ ' (i^)  »ild,-'jP)> 

( d t Cl  ) ^ il  1 ll  I • Q \ ^ 

¥>  -i'--'  • Us)  -iv, 


0(|lE  ||1  . 


BeC4U8e  °f  (3'1H)  an‘‘  »y  >V1U.  th.  s.eond  ,-r»  of  the  len™  tor  t,  and 

'*  sufficiently  small.  finally  we  have 


q’ljq.  l-y  w’cTlw 


(y  - l)  J-  - i ♦ — J Jl — i . 

’ V,  Yt  v. 


<>1  - i) 


YXi 

Y1  S4) 


V) 


uu.’e 


‘I -‘*1  i * w';(:  V I “ 0(  II  w - fiq  . II  ) , 


1 I i j'  '"‘I1 

the  last  part  of  the  lemma  follows  from  il.11),  (1.7)  and  lemmas  1 and  1. 


Lemma  s 

There  are  constants  t > 2n  and  t*  • o such  that 


♦o  - 1 am1  llv*ll  i t1 


imply 


'*  11  j 'H  well-defined  and  positive  definite  for 

111  D*  ( i t , for  j - 0,1,2,... 


0,1,2,... 


il)  )' 


X.-ZII  ■ ><■ 


j-0 


l‘l  oof  . 

Choose  \ - 0.5  in  I.emma  2 and  let  i and  t*  he  such  that 
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d 


(3.20) 


T + 6{^t*  Tj  and  t*  T* 

We  will  show  by  induction  that  then  for  every  j the  following  statements  hold. 

(3-21)  "j+l  *s  we^-<*e^3ned  and  positive  definite  for  i = 0,...,j 

<3-22)  l|xi+1-z||  £ 0.5||x.-z||  , i = 0 j 

j 

(3.23)  1».+1  £ % + 36  l ||x.-z||  < r3  . 

J i=0 

let  j * 0.  Since  (3.20)  implies  T £ T3  £ T2  £ 1£  and  T*  £ t*  £ r*  £ ij,  it  follows 
from  Lemma  3 and  the  definition  of  that  is  positive  definite.  Furthermore,  Lemma  2 

gives  the  inequality  (3.22)  and  Lemma  4 in  conjunction  with  (3.6)  implies  the  relation  (3.23)  . 
Now  assume  that  (3.21)  through  (3.23)  are  satisfied  for  some  j - 1 > 0.  Since  (3.22)  and 
(3.23)  give  the  inequalities. 

IUj-z||  < IU0-z|l  £ t*  and  I (i  < t2 

it  follows  from  Lemma  3 that  H.  . is  positive  definite.  Moreover,  Lemma  2 implies 


lixj+1-zll  £ • 

Using  Lemma  4 and  (3.6)  we  obtain 

*j+1  £ I'j  + 351l|xj-z|) 

3-1 

£ *0  + 3<5i  I llXi'zll  + 36  ||x  -z II 

i=0  J 

j 

— ^0  + 3^i  I (O.S^IIx  -z|| 

i=0 

£ 't'o  + 66lllxo"ZH  - T3  • 

Since  (3.23)  implies  that  the  sum 

! iix  -2n 

j-0  3 

is  finite  the  proof  of  the  lemma  is  complete. 
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Using  the  above  results  we  can  now  prove  that  the  sequence  1x^1  generated  by  the 
algorithm  converges  locally  ami  super  1 ineai ly  to  z. 

Theorem 

l.et  Assumpt  ion  1 be  satisfied.  For  every  choice  of  and  with  ♦ h.  4 0, 

theie  are  numt>eiH 

«(Bj.B2)  and  SMBj.B  > 


such  that  the  inequalities 

II VC  Ml  £ and  ||x0-z||  <_  «S*  ( f<1 . B^) 

imply  that  the  sequences  { x ^ ) and  (li^l  in  the  alqorithm  are  well-defined  and  have  the 
followinq  properties. 

i)  H ^ is  positive  definite  for  all  j. 

ii)  Either  x^  * z for  sixne  j or 

i vnr  • 0 **  1 • ■ 


uu 

I 

j-0 


is  finite 


( H t » and  Ih'1) 


are  bounded. 


Proof ; 


It  follows  immediately  from  Assumption  1 that  there  is  some  neighborhood  U(1(z)  c u(z) 
such  that  X * U0U>  and  x * z imply  VF(x)  * 0.  Choose  a* ( B . < i*  such  that 
||x-z||  >'*(Hj.ti2)  implies  x « ll()(z)  . Furthermore  choose  MBj.B.,1  such  that 


l Ho~G 


-ll 


«(B1.B2) 


implies  ii' 


0 - 


By 


(1./)  a with  this  property  exists  for 

With  *1(Bj»B2)  and  >S*(Bj*B>1  choosen  in  this 
is  well-defined  and  positive  definite  and  x^  t 


every  t > 2n. 

way  we  deduce  from  Lemma  and  Lemma  S that 
U0 ( z ) for  every  1.  Therefore,  it  follows 
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from  Step  1 of  the  algorithm  that  (x^)  la  well-def triad . Furthermore  - 0 if  and  only  if 

* » t. 

) 

Let  ( x ^ 1 he  an  infinite  sequence.  By  l.i'ints  1 aivd  S , the  sequences  (H  ^ 1 and 

(H^)  are  bounded. 

With  0 * t < 1 we  deduce  from  (3.6)  and  l.emma  4 that,  for  every  j, 


( ).  24) 


*1  + l i *j  * ♦ (t-1) 


(*££& a . a 

V Vipi  7 


Since  by  Lemma  l,  ^ ^ 2n  it  follows  from  (1.24)  anti  part  iii)  of  Lemma  5 that 


y . A 

1-0  V Vipi  7 


which  by  (3.7)  implies  that 


(3.25) 


l ||p,q,-Gp.| 

j-0  3 3 


Uainq  (3.12),  (3.25),  (3.14)  and  part  iii)  of  Lemma  5 we  obtain 

y / "-.111'' ' 
i.o  \ Hc'ill 


In  view  of  (3.13)  this  inequality  completes  the  proof  of  the  theorem. 
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